<h1>Results Format</h1>
<p>This page describes the <span class="fontItalic">results format</span> used by COCO. The results format is similar for all annotation types and closely mimics the ground truth format detailed on the <a href="#format-data">data format</a> page. Please review the ground truth format before proceeding.</p>
<p>Each algorithmically generated result, such as an object bounding box or segment, is stored separately in its own <span class="fontItalic">result</span> struct. This singleton <span class="fontItalic">result</span> struct must contains the id of the image from which the result was generated (a single image will typically have multiple associated results). Results for the whole dataset are aggregated in a single array. Finally, this entire <span class="fontItalic">result</span> struct array is stored to disk as a single JSON file (save via <a href="https://github.com/cocodataset/cocoapi/blob/master/MatlabAPI/gason.m" target="_blank" class="fontMono">gason</a> in Matlab or <a href="https://docs.python.org/2/library/json.html" target="_blank" class="fontMono">json.dump</a> in Python).</p>
<p>Example result JSON files are available in <a href="https://github.com/cocodataset/cocoapi/tree/master/results" target="_blank">coco/results/</a> as part of the github package. Because the results format is similar to the ground truth annotation format, the CocoApi for accessing the ground truth can also be used to visualize and browse algorithm results. For details please see <span class="fontMono">evalDemo</span> (<a href="https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocoEvalDemo.ipynb" target="_blank">demo</a>) and also <span class="fontMono">loadRes()</span> in the CocoApi.</p>
<p>The data struct for each of the result types is described below. The format of the individual fields below (category_id, bbox, segmentation, etc.) is the same as for the ground truth (for details see the <a href="#format-data">data format</a> page).</p>

<h1>1. Object Detection</h1>

<p>For detection with bounding boxes, please use the following format:</p>
<div class="json">
  <div class="jsonreg">[{</div>
  <div class="jsonk">"image_id"     </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"category_id"  </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"bbox"         </div><div class="jsonv">: [x,y,width,height],</div>
  <div class="jsonk">"score"        </div><div class="jsonv">: float,</div>
  <div class="jsonreg">}]</div>
</div>
<p>Note: box coordinates are floats measured from the top left image corner (and are 0-indexed). We recommend rounding coordinates to the nearest tenth of a pixel to reduce resulting JSON file size.</p>

<p>For detection with object segments (instance segmentation), please use the following format:</p>
<div class="json">
  <div class="jsonreg">[{</div>
  <div class="jsonk">"image_id"     </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"category_id"  </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"segmentation" </div><div class="jsonv">: RLE,  </div>
  <div class="jsonk">"score"        </div><div class="jsonv">: float,</div>
  <div class="jsonreg">}]</div>
</div>
<p>Note: a binary mask containing an object segment should be encoded to RLE using the MaskApi function <span class="fontMono">encode()</span>. For additional details see either <a href="https://github.com/cocodataset/cocoapi/blob/master/MatlabAPI/MaskApi.m" target="_blank">MaskApi.m</a> or <a href="https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/mask.py" target="_blank">mask.py</a>. Note that the core RLE code is written in c (see <a href="https://github.com/cocodataset/cocoapi/blob/master/MatlabAPI/private/maskApi.h" target="_blank">maskApi.h</a>), so it is possible to perform encoding without using Matlab or Python, but we do not provide support for this case.</p>

<h1>2. Keypoint Detection</h1>
<div class="json">
  <div class="jsonreg">[{</div>
  <div class="jsonk">"image_id"     </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"category_id"  </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"keypoints"    </div><div class="jsonv">:  [x<sub>1</sub>,y<sub>1</sub>,v<sub>1</sub>,...,x<sub>k</sub>,y<sub>k</sub>,v<sub>k</sub>],</div>
  <div class="jsonk">"score"        </div><div class="jsonv">: float,</div>
  <div class="jsonreg">}]</div>
</div>
<p>Note: keypoint coordinates are floats measured from the top left image corner (and are 0-indexed). We recommend rounding coordinates to the nearest pixel to reduce file size. Note also that the visibility flags v<sub>i</sub> are <i>not</i> currently used (except for controlling visualization), we recommend simply setting v<sub>i</sub>=1.</p>

<h1>3. Stuff Segmentation</h1>
<div class="json">
  <div class="jsonreg">[{</div>
  <div class="jsonk">"image_id"     </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"category_id"  </div><div class="jsonv">: int,  </div>
  <div class="jsonk">"segmentation" </div><div class="jsonv">: RLE,  </div>
  <div class="jsonreg">}]</div>
</div>
<p>The stuff segmentation format is identical to the object segmentation format except the score field is not necessary. Note: We recommend encoding each label that occurs in an image with a single binary mask. Binary masks should be encoded via RLE using the MaskApi function <span class="fontMono">encode()</span>. For an example see <span class="fontMono">segmentationToCocoResult()</span> in <a href="https://github.com/nightrome/coco/blob/master/PythonAPI/pycocotools/cocostuffhelper.py">cocostuffhelper.py</a>. We also provide <a href="https://github.com/nightrome/coco">conversion scripts</a> between the JSON and png formats for convenience.</p>

<h1>4. Panoptic Segmentation</h1>
<p><b>Details coming soon!</b></p>

<h1>5. Image Captioning</h1>
<div class="json">
  <div class="jsonreg">[{</div>
  <div class="jsonk">"image_id"     </div><div class="jsonv">: int,      </div>
  <div class="jsonk">"caption"      </div><div class="jsonv">: str,      </div>
  <div class="jsonreg">}]</div>
</div>
